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How to improve it 

The Dl's Product Review 
Process 
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". . .// is worth remembering that the basic purpose 
of the review process is to take the individual ideas 
or judgments of a single analyst or a few analysts, 
and turn those ideas or judgments into the institu- 
tional position of the Central Intelligence Agency. " 

DDI Newsletter 
12 January 1984 



From the standpoint of intelligence analysis 
per se, the product review process is one of 
the central fixtures of professional life in the 
Directorate of Intelligence (DI). It is also an aspect 
of intelligence work that deserves more attention and 
study than it has traditionally received. While atti- 
tudes toward review vary tremendously across the 
directorate, most observers agree that the process 
could be improved. The last Dl-wide survey on the 
subject, conducted in late 1988 by the DI Manage- 
ment Advisory Group (MAG), found that the biggest 
single complaint from analysts was not with the 
process itself but with management's failure to fol- 
low the prescribed process. I would argue that as a 
directorate we do not have any "prescribed" process, 
and that many of the problems with review can be 
traced to this basic lack of a conceptual framework. 

For new analysts, review can be an unexpected, 
humbling experience that seems to contradict many 
of the norms of their academic experience. In the 
case of complex or contentious pieces review can be 
a grueling exercise, even for experienced analysts. In 
some cases, the outcome of the process may be a de- 
cision not to publish, with months of research and 
writing seemingly wasted. One of the most common 
grievances expressed by analysts is that there are too 
many layers of review, leading to contradictory re- 
quirements and redundant or "circular" revisions. 
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Managers have to deal with similar problems, though 
from a different perspective. Their greatest challenge 
is often to mediate conflicting positions and produce 
compromise language without gutting the analysis. 
And, finally, for all involved the process can some- 
times be protracted and exhausting. Many of us have 
heard stories about items for the National 
Intelligence Daily (NID) that took a week to coor- 
dinate or "hardcover" papers that took a year from 
first draft to final publication. 

In my view, the existing review process is necessary 
and it is not fundamentally flawed. Many of its 
problems can be ameliorated, if not solved, by build- 
ing a clearer shared understanding of its goals and 
purposes. 



First Things First 

The directorate has no succinct, common definition 
of review. Some DI offices have statements about 
how review is supposed to function. But these are 
usually mechanistic descriptions or production 
timetables detailing, for example, how much turn- 
around time each management echelon has for a 
paper. Because of differences in culture and subject 
matter, and the latitude given office directors by the 
Deputy Director For Intelligence, at this level the 
process can differ substantially from office to office. 

In general terms, review is the contribution of the or- 
ganization to the production of a particular intelli- 
gence product, as opposed to the contribution of the 
individual analyst. (This discussion deals primarily 
with the production of longer-term written intelli- 
gence products, though some if it applies to current 
pieces and high-level briefings.) This may seem ob- 
vious, but it is an important characterization because 
it reminds us that intelligence in whatever form it 
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may take is essentially an amalgam of individual and 
organizational efforts, both of which have to be 
given their due. To depreciate the individual's contri- 
bution is to forget that all analysis has to begin with 
some person's intellectual effort; ignoring the organi- 
zation's rightful role risks straying into the morass of 
analytical pride. 

At a more specific level, review can be seen as hav- 
ing four basic elements: coordination, validation, 
editing, and production. These are not, however, se- 
quential stages; they are logically, but not necessarily 
temporally, distinct. Coordination, which some mis- 
takenly assume to be synonymous with review, is the 
quality-control part of the process. It consists of the 
examination of a particular piece of intelligence by 
others in the organization to eliminate errors of fact, 
to identify additional perspectives that might not 
have occurred to the author, and to ensure that the 
judgments drawn are congruent with those taken by 
other DI components. 

Validation is the adoption by the organization of the 
analysis contained in the product as its corporate po- 
sition. It is the heart of the process. Validation for- 
mally occurs when the product receives final ap- 
proval from the appropriate management level, but it 
actually takes place throughout the process, begin- 
ning with management's decision to undertake the 
project. Through the sometimes innumerable coordi- 
nation meetings and draft rewrites, the organization 
gradually takes "ownership" of the product from the 
analyst. This is often difficult for both parties: the 
analyst has to learn to give up total control of sub- 
stance, while the organization has to take care not to 
muddle or compromise the analytical message. Even 
after final approval, some may remain skeptical of a 
piece's analytical line. There is nothing wrong in 
this. When the Agency puts its seal on a product, it 
signifies only an organizational consensus, not a una- 
nimity of opinion. In cases where opposing views are 
held by a significant minority, differences can and 
should be reflected in the draft itself. 

Broadly speaking, the goals of the editorial stage of 
review are to improve the readability and organiza- 
tion of the text and bring it into conformance with 
the DI writing style. Good editors are able to im- 
prove the clarity of a piece while leaving substance 
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unaffected. Because editing plays a role in refining 
the presentation of analysis, a critical aspect of intel- 
ligence production, it is properly considered part of 
review. Unfortunately, a basic problem with editing 
is that too many in the process view it as their 
responsibility, a fact that often leads to an unneces- 
sary duplication of. effort. For example, how many of 
us can recall expending considerable effort on the 
precise wording of a piece of current intelligence at 
the office level and below, only to have the NID 
Staff substantially reword it? 

The final phase of the review process is usually 
dominated by production, during which the piece is 
prepared for publication by the Office of Current 
Production and Analytic Support production staff, 
the author, and his or her home office editorial staff. 
Although some might consider it inappropriate to in- 
clude production as part of review, important deci- 
sions regarding the overall layout, the presentation of 
maps, figures and other visual aids, and the finaliza- 
tion of the piece's dissemination list are made only 
at this stage. 

Perspectives on the Process 

A majority in the DI regardless of position, seniority, 
or substantive speciality believe the review process 
is imperfect. In my experience, observers of it 
usually fall into one of three general categories. The 
first could be labeled the "pessimist" school. Its 
proponents argue that the existing process is systemi- 
cally corrupt and that no amount of tinkering will 
cure its ills. For example, they cite the fact that those 
closest to "substance," the line analysts, have a 
weaker voice in deciding disputed intelligence judg- 
ments than their non-expert superiors. This, they 
sometimes hold, opens the doors to the politicization 
of intelligence. Others claim that minority positions 
are often submerged by the dominant corporate view, 
and that fresh, groundbreaking analysis cannot occur 
because of the demands of building consensus. In 
view of the seriousness of these shortcomings, they 
argue, the existing edifice of review should be razed 
and rebuilt. 

A second, more mainstream group might be dubbed 
the "cynics." While disagreeing with the pessimists' 
contention that the process in inherently flawed, this 
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group agrees that it is a waste of time to try to im- 
prove it. Cynics have witnessed many attempts to 
"fix" the review process, most of which they judge 
to have had little practical effect. They are, therefore, 
content to live with the prevailing process. It is 
difficult to generalize about the predominant atti- 
tudes of this group because they cover such a wide 
spectrum. Most seem to believe, however, that the 
shortcomings of review are a natural consequence of 
the conflict between the individual's desire for sub- 
stantive creativity and judgmental freedom and the 
organization's demand for conformity and com- 
promise. If the cynics had a motto, it probably would 
be: "When you can fix human nature, then you'll fix 
the review process." In an extreme form, some cyn- 
ics have almost a "sink or swim" mentality: having 
put up with and succeeded in spite of the challenges 
posed by the review process, they see no reason why 
others, particularly new analysts, cannot do so as 
well. 

I call the last group the "optimist" school. It be- 
lieves that the process is not fundamentally defective 
and that its deficiencies can be remedied to some 
significant degree. The mark of an optimist is the 
willingness to entertain and at times to implement 
significant changes or experiments in the review 
process. Most are cautious, however, because they 
recognize that tinkering with a basically sound 
process has its risks. In a perverse way, the optimists 
in the DI probably are responsible for most of the 
shortcomings of the process because they have been 
in the vanguard over the years in refining the system. 
But this should not deter us from suggesting or at- 
tempting solutions. 



Tackling the Problem 

To my mind, all of the problems with the review 
process in the DI boil down to two: one substantive 
and one perceptual. The substantive problem is that 
the functions of the individuals involved in the 
process are not drawn clearly enough. This leads to 
a duplication of effort and unnecessary delays. What, 
for example, is the difference between the functions ' 
of the branch chief and the division chief? Should 
there be any difference? Among peers, where does 
the burden of persuasion lie with regard to disputed 
judgments, with those requesting change or with 
those resisting change? What is the threshold for 
including dissenting opinions in publications? In the 
case of editing, where does the branch chief's role 



end and the office editorial staff's begin? All of us 
have different answers to these questions but no 
common set of guidelines. 

This problem could be substantially reduced if each 
of the participants in the process concentrated on that 
function that they perform best. For example, 
analysts are best suited to perform research and draft 
basic analysis. Branch chiefs have greater ex- 
perience, and they are still in touch with the subject 
matter on a routine basis; they are best equipped to 
perform a thorough substantive check. Higher-level 
managers have greater experience, and they are more 
in touch with consumers; they are best able to exa- 
mine the product for focus and policy relevance. And 
so it goes with all the other participants. Instead of 
having the perspective of producers who "assemble" 
a product, with each contributing his or her share to 
the as yet imperfect whole, we often seem more like 
artists who feel that the work of art can only go for- 
ward in "perfect" form (hence the stylistic editing at 
every level of review). We need more producers and 
fewer artists. 

The perceptual problem with the review process is 
that each one of us defines it in a different way. Some- 
times these differences are significant. The particulars 
of the process vary widely across the DI, and that is 
as it should be. But there exists no authoritative frame- 
work within which to structure the details. In the 
MAG survey, analysts felt most frustrated that manage- 
ment did not follow the prescribed process. This is a 
direct reflection of the varying expectations between 
analysts and managers about what review is and how 
it should function. This has sometimes contributed to 
a perception among analysts that, while they are held 
accountable for following the prescribed process, 
management has no such strictures. 

When analysts join the DI, they are expected to be 
able to do research, analyze, and write. Such expec- 
tations are reasonable because these skills are tested 
on the PATB and because they are required to make 
it through the university system. It is curious, 
however, that we also seem to expect them to under- 
stand the review process without ever having been 
exposed to it. In the Office of Training and 
Education's (OTE) Analyst Training Course, a great 
deal of effort is rightly devoted to teaching new 
analysts how to write in the DI style. But the review 
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process receives little attention, even though an un- 
derstanding of it will be as important to an analyst's 
success as mastering the DI writing style. Similarly, 
in the Supervision of Analysis Course, new 
managers are taught how to review papers but not 
about the review process itself. OTE is not responsi- 
ble, however, for this state of affairs; the DI has no 
formal review concept that could serve as a basis for 
such instruction. 

Recommendations for Change 

As a first step, the DI should develop and promul- 
gate a directorate-level product review process con- 
cept, with which all individual office review proce- 
dures must be in accordance. The broad outlines of 
such a concept have already been defined, most re- 
cently in the DI Notice Guidelines for Product 
Review, Coordination, and Incorporation of 
Alternative Views (DI N 20-205, 2 June 1992). They 
now need to be developed further and formally pub- 
lished in a manual or similar document. This effort 
should involve both managers and analysts, and it 
should focus on elaborating an overarching concept 
of review that can be applied across all of the DI 
line offices, rather than establishing particular review 
procedures, which should be left up to the individual 
offices. 

Once developed, this concept should be integrated 
into OTE training for analysts and managers in the 
DI. For analysts, this could be accomplished primar- 
ily through a block of instruction in the Analyst 
Training Course, which is required for all new 
analysts. It could also be presented in the Workshop 
for DI Midlevel Analysts and the DI Writing 
Workshop. Managers could receive instruction during 
the Supervision of Analysis Course, which is manda- 
tory for all new branch chiefs. The DI review 
process concept could also be included in an analysis 
tradecraft manual, development of which was called 
for in another recent DI Notice. Over time, the con- 
cept would permeate the directorate and become its 
institutional philosophy. And, unlike the current situ- 
ation, it would be explicit and commonly held. If ex- 
perience so warrants, the concept could and should 
be modified as conditions change. 



Toward a DI Review Philosophy 

The DCI's task force on Politicization and 
Intelligence Production has already taken steps to ad- 
dress many of the concerns raised here. Their efforts 
are a good start toward developing a more clear-cut 
philosophy of review in the DI. Some of my 
thoughts on the possible substance of such a 
philosophy follow: 

— The central purpose of review is to produce a 
product that represents the institutional position 
of the CIA. 

— All other things being equal, fewer levels of 
review are better than more. There should exist a 
presumptive requirement to demonstrate that a 
new level of review will significantly improve 
the product before it is created. 

— Review begins at the conceptual stage of a paper, 
not when analysis and drafting are completed, 
and it carries through to the final physical 
production stage. 

— For the analyst, review is as much about giving 
up his or her sole control of the product as it is 
about getting others to take ownership of it. 

— Even though the analyst plays the primary role in 
carrying out the day-to-day procedures of review, 
management should bear the ultimate responsibil- 
ity for the success or failure of the process. 

— All parties should be held accountable to the es- 
tablished review process, except in the most un- 
usual circumstances. 

— There is no room for ego or literary hubris in the 
review process. Intelligence production in the DI 
is a team effort in which compromise, next to the 
pursuit of truth, is the most highly valued com- 
modity. 

— Review is logically distinct from research but not 
from analysis. The intellectual scrutiny to which 
products are exposed during the process is an in- 
tegral part of the final published analysis. 
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— Greater emphasis should be put on timeliness. 
According to the recent DCI task force report on 
production, consumers believe that timeliness is 
sometimes a problem with DI products, and they 
generally value it more highly than quality of 
content. 

— To reduce duplication of effort and enhance time- 
liness, participants in the process should concen- 
trate on the task or function that they alone 
among the participants are best able to perform. 
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